Joint Synthesis of Safety Certificate and Safe Control Policy using Constrained Reinforcement Learning

Safety is the major consideration in controlling complex dynamical systems using reinforcement learning (RL), where the safety certificate can provide provable safety guarantee. A valid safety certificate is an energy function indicating that safe states are with low energy, and there exists a corresponding safe control policy that allows the energy function to always dissipate. The safety certificate and the safe control policy are closely related to each other and both challenging to synthesize. Therefore, existing learning-based studies treat either of them as prior knowledge to learn the other, which limits their applicability with general unknown dynamics. This paper proposes a novel approach that simultaneously synthesizes the energy-function-based safety certificate and learns the safe control policy with CRL. We do not rely on prior knowledge about either an available model-based controller or a perfect safety certificate. In particular, we formulate a loss function to optimize the safety certificate parameters by minimizing the occurrence of energy increases. By adding this optimization procedure as an outer loop to the Lagrangian-based constrained reinforcement learning (CRL), we jointly update the policy and safety certificate parameters and prove that they will converge to their respective local optima, the optimal safe policy and a valid safety certificate. We evaluate our algorithms on multiple safety-critical benchmark environments. The results show that the proposed algorithm learns provably safe policies with no constraint violation. The validity or feasibility of synthesized safety certificate is also verified numerically.Comment: 24 pages, 8 figures, accepted for oral presentation at L4DC 202

What Truly Matters in Trajectory Prediction for Autonomous Driving?

In the autonomous driving system, trajectory prediction plays a vital role in ensuring safety and facilitating smooth navigation. However, we observe a substantial discrepancy between the accuracy of predictors on fixed datasets and their driving performance when used in downstream tasks. This discrepancy arises from two overlooked factors in the current evaluation protocols of trajectory prediction: 1) the dynamics gap between the dataset and real driving scenario; and 2) the computational efficiency of predictors. In real-world scenarios, prediction algorithms influence the behavior of autonomous vehicles, which, in turn, alter the behaviors of other agents on the road. This interaction results in predictor-specific dynamics that directly impact prediction results. As other agents' responses are predetermined on datasets, a significant dynamics gap arises between evaluations conducted on fixed datasets and actual driving scenarios. Furthermore, focusing solely on accuracy fails to address the demand for computational efficiency, which is critical for the real-time response required by the autonomous driving system. Therefore, in this paper, we demonstrate that an interactive, task-driven evaluation approach for trajectory prediction is crucial to reflect its efficacy for autonomous driving

An Effective Data Sharing Scheme Based on Blockchain in Vehicular Social Networks

Vehicular social networks (VSNs) are the vehicular ad hoc networks (VANETs) that integrate social networks. Compared with traditional VANETs, VSNs are more suitable to serve a group of vehicles with common interests. In VSNs, vehicles can upload the necessary data in the cloud service provider (CSP) and other vehicles can query the data they are interested in through CSP, which enables VSNs to provide more user-friendly services. However, due to the wireless network communication environment, the data sent by the vehicle can easily be monitored. Adversaries are able to violate the privacy of the vehicle based on the collected data, thereby threatening the security of the entire network. In addition, if a vehicle shares malicious or false data with other vehicles, it is easy to mislead drivers and even cause serious traffic accidents. This paper proposes an effective data sharing scheme based on blockchain in VSNs. By integrating an identity based signature mechanism and pseudonym generation mechanism, we first propose an anonymous authentication mechanism as the basis for establishing trust relationships before data transmission between entities in VSNs. Then, a data sharing scheme based on blockchain is described, in which the signature mechanism and the consensus mechanism guarantee the security and traceability of data. The result of the performance analysis and the simulation experiment indicate that VAB can achieve a favourable performance compared with existing schemes

Design of a loudspeaker array for personal audio in a car cabin

With an increasing interest in personal audio systems, the car cabin is an important environment in which to generate different audio programs in different regions, without the use of headphones. Two algorithms, acoustic contrast control and the least squares method, are considered for producing two independent listening zones, one zone including the front passengers and the other including the rear passengers. The generation of an acceptable acoustic contrast between the front and rear zones, using an array of four standard audio loudspeakers, is limited to relatively low frequencies. In order to provide acceptable control over a larger audio bandwidth, a loudspeaker array mounted on the ceiling of a car cabin is investigated in this paper. A selection process for the configuration of the source array is described. Free field simulations are used to calculate the response of the source array and investigate the performance of the two control algorithms. Since the performance of the least squares method is dependent on the chosen target sound pressures, a method for selecting the target sound pressures is also proposed. Finally, the proposed loudspeaker array is implemented in a real car and the measured results are found to be similar to those predicted from the simulations

Real-Time Center of Gravity Estimation for Intelligent Connected Vehicle Based on HEKF-EKF

The vehicle center of gravity estimation is the key technology to the vehicle active safety system in intelligent connected vehicles. In this study, an integrated estimation approach for center of gravity (CG) combining Huber Extended Kalman Filter and Extended Kalman Filter (HEKF-EKF) is proposed. First, HEKF algorithm is used to estimate the distance between the CG and the front axle at the current time. Then, the CG height obtained by HEKF and EKF algorithms is weighted to obtain the optimal estimate value. Finally, the results show that the algorithm’s estimation convergence time is 2 s, its longitudinal position estimation error is less than 2%, and its center of gravity height estimation error is less than 3%. The longitudinal and vertical positions of the vehicle CG can be accurately estimated using this method. This method can help advance the development of active safety technology

Zeroth-Order Actor-Critic

The recent advanced evolution-based zeroth-order optimization methods and the policy gradient-based first-order methods are two promising alternatives to solve reinforcement learning (RL) problems with complementary advantages. The former methods work with arbitrary policies, drive state-dependent and temporally-extended exploration, possess robustness-seeking property, but suffer from high sample complexity, while the latter methods are more sample efficient but are restricted to differentiable policies and the learned policies are less robust. To address these issues, we propose a novel Zeroth-Order Actor-Critic algorithm (ZOAC), which unifies these two methods into an on-policy actor-critic architecture to preserve the advantages from both. ZOAC conducts rollouts collection with timestep-wise perturbation in parameter space, first-order policy evaluation (PEV) and zeroth-order policy improvement (PIM) alternately in each iteration. We extensively evaluate our proposed method on a wide range of challenging continuous control benchmarks using different types of policies, where ZOAC outperforms zeroth-order and first-order baseline algorithms

Effect of cesium ion on the synthesis and catalytic properties with FeCo Prussian blue analogue

International audienceIn this article, cesium ions were introduced into the FeCo Prussian blue analogues (PBA) catalyst to increase the styrene conversion and the selectivity in styrene oxide in the reaction of epoxidation of styrene. The conversion of styrene over CsFeCoPBA-1 on the optimal condition reach up to 96%, in comparison to 88% over FeCoPBA. The catalysts were characterized by fourier transform infrared spectroscopy, X-ray diffraction, transmission electron microscopy (TEM), X-ray energy dispersive spectroscopy (EDS), thermogravimetric analysis (TG) and N2 adsorption/desorption, respectively. To further illustrate the phenonmen, the detailed kinetic studies on epoxidation of styrene were carried out. It is found that the rate equations is similar between the catalyst CsFeCoPBA-1 and FeCoPBA, while the reaction activation energy with CsFeCoPBA-1 catalyst is reduced obviously. Meanwhile, isothermal titration calorimetry (ITC) had also been employed to directly probe thermodynamic changes during the formation of the PBA with or without Cs ions. It is noteworthy that higher enthalpy change in the presence of cesium ions